\section{Introduction and related Work}
In the recent years Deep Neural Networks and in particular Convolutional Neural Networks (CNNs) have been established as state of the art techniques for image classification and object detection ~\cite{szegedy2015going,rajpura2017object}.
Current state of the art object classification methods such as \textit{Faster R-CNN} and \textit{SSD} are widely used and provide good results in real-time object classification.
Another approach is \textit{You Only look Once} (YOLO), as presented in \cite{redmon2016you}, in which a single neural network predicts object classes and bounding box positions in one detection iteration.
In the latest iteration, \textit{YOLOv3}~\cite{redmon2018yolov3}, Redmon and Farhadi showed that their method provides a significant speed improvement compared to SSD and Faster R-CNN.
This speed-up allows the application of object detection to real-time applications.

However, training neural networks for object classification tasks requires vast amounts of training data, which is usually difficult to obtain because it requires human generated annotations.
Labeling images is a laborious and time intensive task and is a major obstacle when designing new applications based on object detection and classification.
In recent research domain randomization is used in simulation for training DNNs and then transferring them to real world applications~\cite{tobin2017domain}.
Further research has investigated how synthetic images can be used to train CNNs~\cite{rajpura2017object,prakash2018structured}.
Rajpura et al. found that synthetic images created from rendering 3D-models in a randomized 3D-scene provide less precise object detection, but when combined with real data lead to a higher mean average precision (mAP).
In a recent work Prakash et al. proposed a new method for synthetic dataset generation that takes the structure and the context of a scene into account.
Trained to detect cars using only synthetic data and benchmarked on the KITTI benchmark suite, the method outperformed other state of the art synthetic data generation methods.
Furthermorel, the paper supports the claim that combining synthetic and hand labeled data further improves the performance of the trained model.

It is evident that the capability to automatically generate training data for new domains can drastically reduce the effort required to train new models and allow for more flexible and efficient applications using object detection.

In this paper I present a method to generate large synthetic randomized datasets with precise labels by generating two dimensional masked raw data from 3D models of objects, combining them with backgrounds from the game to generate training data.
Finally different levels of noise and randomization are applied to the images to make the training data more variable and thus the model more robust.
The dataset generation pipeline and the benefits of large and randomized datasets versus hand labeled datasets are demonstrated by training different \textit{YOLOv3} object classifiers for the video game \textit{League of Legends} and comparing their mean average precision (mAP).
Furthermore the detection precision of the trained model is compared to a model trained on hand labeled data and to a model trained with both hand labeled and synthetic data.

The results show that a model trained with the synthetic data is capable of achieving significantly higher mAP with more detectable classes while taking only a fraction of the time it takes to label the images by hand.
Combining synthetic and hand labeled data did not further improve mAP as a result of differing image aspect ratios of the older hand labeled dataset.
The detection speed allow real-time use of the detectors at roughly 12-18 frames per second on a Nvidia GTX1080 GPU while running the game on the same machine.

\section{Code}
In this section the structure of the code of this project is briefly explained.
All snapshot of the currently private repository is available on \href{https://github.com/Oleffa/LeagueAI}{GitHub}\footnote{https://github.com/Oleffa/LeagueAI}.
\begin{itemize}
\item \texttt{LeagueAI\_helper.py}: The core of the framework providing all important functions for object detection, loading datasets etc.
\item \texttt{LeaugeAI\_minimal\_example.py}: A minimal example demonstrating how to use the framework with different inputs such as videos, images and in real-time on the game.
\item \texttt{LeagueAI\_vayne\_bot.py}: An implementation of the bot from the old project with the new LeagueAI framework.
\item \texttt{LeagueAI\_mAP.py}: Compute the mean average precision for each class against a test dataset.
\item \texttt{plot\_loss.py}: Plot the loss during training.
\item \texttt{yolov3\_detector.py}: The YOLOv3 detector implemented from scratch following this tutorial: \href{https://blog.paperspace.com/how-to-implement-a-yolo-object-detector-in-pytorch}{How to implement a YOLO object detector in pytorch}
\end{itemize}
The code for the dataset generation is in the subfolder \texttt{generate\_dataset/}.
Next to the dataset generation script itself, various helper scripts are included:
\begin{itemize}
\item \texttt{pyFrameexporter.py}: Export frames from videos
\item \texttt{pExportTransparentPNG.py}: Export cropped and masked PNG files of the individual objects from a uni-color background
\item \texttt{rename\_classes.py}: Rename classes within the labels of a dataset (used to combine multiple classes into one, or when the ID of an object changes)
\item \texttt{split\_test\_train.py}: Dividing a dataset in training and testing dataset
\item \texttt{remove\_outline.py}: A script to manipulate the outline of objects to add outlines, remove outlines and add glow around objects
\item \texttt{convert\_tensorflow\_to\_pytorch.py}: Convert Tensorflow label data to the format required to train YOLOv3
\item \texttt{check\_dataset\_integrity.py}: A script to check if a label exists for every image
\end{itemize}
